This patent application includes microfiche Appendix A which is a part of the present disclosure and which is incorporated by reference herein in its entirety. This Appendix consists of a total of 2 sheets that contain a total of 122 frames. Appendix A is a listing of software code for embodiments of the present invention, which are described more completely below.
1. Field of the Invention
The present invention relates generally to computer networking and, in particular, to a system that performs clustering of routers in multiple network servers.
2. Discussion of Related Art
Increasing numbers of companies are conducting transactions and building relationships online, with consumers and business partners in a phenomenon called xe2x80x9ce-business.xe2x80x9d
E-business is usage of Internet-based technologies to improve communications with customers and information exchange with business partners, and to increase sales. Web sites, electronic commerce, intranets, and extranets are all examples of E-business entities. Many Web, DNS, proxy, FTP, firewall, and application server services are needed in any of these environments. If any of the services fails or responds slowly, the entire system suffers. Downtime in today""s highly competitive marketplace can mean lost revenue and lost credibility.
Adding more memory or substituting a larger server can expand the capacity of an E-business entity, such as a Website. In either case, expansion requires downtime and an additional capital expenditure. In conventional systems, the only way to supply a reliable capacity to meet a rapidly increasing demand is to purchase computing power far beyond that which meets current needs. Then, once the demand exceeds the purchased computing power, the entire computer must be replaced and the purchase cycle repeated. The conventional business practice is highly inefficient, wasteful of operating expenses, and inadequate for meeting customer needs.
One technique for improving the reliability and responsiveness of Internet sites is the usage of server clusters. Server clusters are arrays of servers that share common tasks while appearing to external devices as single resources. In a cluster, each server shares the load that would be handled traditionally by a large server alone. A user accesses a cluster in the same manner as a single server is accessed.
Clusters improve both reliability and scalability of operation in comparison to single server operation. Redundant servers in a cluster support tolerance to system failures.
Several basic approaches to Internet server clustering and load balancing are presently available. One approach is the usage of software executing on one or more servers to create clusters of Internet Servers. The clusters are peer-to-peer clusters with no arbitrator or manager. According to measurements using benchmarking tools, software-based cluster performance is generally poor since servers in the cluster must devote significant amounts of computer power to managing the cluster. Also, communications among the servers for cluster management and resource arbitration add large amounts of data onto the network connecting the servers.
A second approach is the usage of xe2x80x9cload balancingxe2x80x9d software executing on special-purpose industrial PCs. A PC executing the load balancing software broadcasts the source PC Internet Protocol (IP) address as an identification address of the cluster of Internet Servers. As network traffic is received, the load balancing system uses a technology known as Network Address Translation (NAT) to modify each packet, changing from the source IP address to the actual address of the server that is to receive the traffic. The server responds to the request and the PC load balancing software again uses NAT to modify the xe2x80x9cFromxe2x80x9d address in each packet to create the appearance that the PC load balancer sent the packets. PC load balancers are restricted to applications that use TCP/IP (transmission control protocol/Internet protocol), effectively limiting the applications to technologies including Web servers, firewall servers and proxy servers. The TCP/IP products usually become a bottleneck for data traffic into and out of the cluster, slowing overall performance. NAT technology requires that the body of each packet be available to the processor, so a cluster based on a PC load balancer cannot be used if the data packets are encrypted.
A third approach is the usage of load balancing software that executes from a local area network (LAN) switch rather than a PC. Execution from the switch hardware increases data transfer rates but remains based on NAT technology using TCP/IP, limiting applications to Web servers, firewall servers, and proxy servers. Switch-based and PC load balancers cannot be used when the data packets are encrypted.
What is needed is a system and operating method that attains very high data throughput rates and supports all Internet protocols.
A router clustering system connects two or more routers to one or more distinct Internet Service Providers (ISPs) in a high-availability arrangement. Typically, the two or more routers connect to a plurality of ISPs at a site so that an outage of one ISP does not affect connectivity and availability to the site. The router clustering system typically includes a plurality of clustering units for redundancy that avoids difficulties that arise with a single point of failure. Fore example, two clustering units may be used in an active-passive high-availability configuration.
Various implementations of the router clustering system have several router clustering features and benefits. Outgoing traffic that is destined for devices on the Internet is distributed among the two or more routers included in the cluster. The router clustering system distributes traffic based on the destination IP addresses of the packets thus ensuring that all IP-based protocols are supported.
In some embodiments, the same router handles all outgoing traffic for a single client-server session, and unlimited client-server sessions can be supported.
In some embodiments, the same router handles all outgoing traffic for a single client-server session and unlimited client-server sessions can be supported.
For communication interconnects using a router clustering system, servers need not be configured with multiple gateways. Servers are configured to use a single xe2x80x98logicalxe2x80x99 gateway having an IP address identifying the router cluster.
In some implementations, the router clustering system continually monitors the operational health of the routers and associated wide area network (WAN) links.
In some implementations, the router clustering system detects one or more of various failure conditions including: (1) failure of the router LAN interface and link, (2) failure of the router due to power outage, software malfunction, hardware malfunction, or other condition, and (3) failure of the router WAN interface and link. When the router clustering system detects a failure, traffic is automatically forwarded to the remaining operational router or routers. The router clustering system does not require manual intervention at the server to bypass the failed router.
Additional routers may be seamlessly added to supply additional bandwidth and greater fault tolerance.
The router clustering system operates in a manner that is independent of the router hardware and software. Various combinations of routers can exist in the cluster as long as the routers have the same connectivity.
In one aspect of a router clustering system, a router cluster creator creates or configures a router cluster. To create a router cluster, an administrator assigns to the cluster a logical Internet protocol (IP) address IPgw and specifies routers, Router1 and Router2, that are members of the cluster. Typically the member routers are previously configured as servers. The router clustering system begins to monitor xe2x80x9chealthxe2x80x9d of the routers. In some implementations, the router clustering system monitors router health using a configured polling interval and health check method.
In some implementations, a logical router cluster address IPgw is configured on the servers as a xe2x80x98defaultxe2x80x99 gateway rather than a unique IP address of one of the routers.
The router clustering system responds to an Address Resolution Protocol (ARP) request from the servers to identify a Media Access Control (MAC) address associated with the router cluster. Associating the MAC address with the router cluster ensures that the servers send all outbound traffic to the router clustering system for forwarding on to the designated final destination on the Internet.
In another aspect of the router clustering system, a traffic distributor distributes outbound traffic destined for the Internet among the routers. The traffic distributor of the router clustering system determines which router is to forward the outbound traffic based on the packet destination IP address. Usage of the destination IP address ensures that, for a given flow designating a server-destination connection, the same router is used for every outbound packet so long as the router remains operational. Internally, the traffic distributor maintains a list of operational routers. Fields from the packet are used to compute the index into this list, identifying the active router.
For other types of clusters that are supported, the traffic distributor only modifies the destination MAC address of the redirected traffic. The traffic distributor is a software process that executes on hardware devices such as a LAN switch. A hardware device has a particular MAC address that identifies the traffic distributor. The traffic distributor replaces the packet destination MAC address, which previous to replacement is the MAC address of the traffic distributor, with the MAC address of the router handling the flow.
Each router has an equal probability of assignment for an outbound flow forwarding since the traffic distributor uses only information in the packet IP header to select between routers. Processing load or potential processing power of the router is not analyzed in the selection.
The router cluster does not affect the processing performed by the router clustering system for inbound traffic coming from the Internet. Traffic destined for any of the server clusters continues to be distributed among the operational servers defined in the server clusters. Multiple firewall or router clusters are supported for inbound traffic (e.g. by use of multiple MAC addresses).
A clustering system operates on all types of Internet protocol (all/IP) technologies and can be used to create a cluster of any Internet Servers, no matter what protocol is running on IP, even Voice over Internet protocol (Volp) and streaming audio/video via User Datagram Protocol (UDP/IP).
The clustering system avoids problems associated with NAT such as an inability to encrypt the data, because the all/IP approach allows each of the servers in the cluster to use the same IP address as the cluster""s overall address.
In some embodiments, the clustering system executes on local area network (LAN) switch hardware to attain very high data throughput rates.
Unlike Switch-based load balancers, a clustering system does not process packets flowing from servers to users, the direction of the largest data flow. Instead, the router clustering system operates as a wire-speed switch for downstream traffic.
Advantages of a clustering system depend on the particular implementation of the system. One advantage is that capacity of the cluster increases linearly as additional servers are added to the cluster. In various implementations, the clustering system manages all or some of the cluster activities, freeing servers in the cluster from expending computing power on cluster management.
The clustering system controls connections of clients to particular servers, reducing the computing required to manage the cluster on servers in the cluster and freeing computing power to be applied to the task of the cluster.
The clustering system can manage many different clusters simultaneously, allowing specific hardware to easily migrate from one cluster to another, as demand patterns dictate.
Management of a clustering system and one or more clusters is accomplished through any of several different management methods, including telnet, CLI, Web browser, and SNMP. The clustering system assists customers with an easy-to-use single point of management to control multiple clusters in multiple tiers of computing. The clustering system allows administrators to choose the management method that works best for the particular business characteristics and facility.